The present invention relates to an error concealment for a digital video signal to conceal an error generated from the block-based video coding system such as Moving Picture Experts Group(MPEG) system.
In case of transmitting the compressed digital video signal via some channel, usually an error is generated on a transmission channel and this error causes a loss of image data during reconstruction of the video signal at the receiver side.
As a method for correcting such error, typically an error correcting code is sent, for example, a parity bit to the digital video signal. In accordance with this, it enables to reconstruct the picture by decoding matched to the error correcting code However, there is a limitation to the number of the correctable errors.
Accordingly, in case when any error is judged as, being uncorrectable at the decoder due to this limitation, an additional operation such as the error concealment.
As a typical method for the concealment of such an error, there is a method of substituting the error part by interpolating either using the neighborhood pixels within the same frame of by using the neighborhood pixels of its post frame. As an example of this a patent application filed and laid opened in Great Britain by SONY Corp. of Japan is described in the G.B.P. laid open publication No. 2.163,619A under the title of "Error concealment in digital television signal", and this is shown in FIG. 1 to FIG.3.
FIG. 1 is a schematic block diagram of a digital television signal reproducing system described in the aforementioned English patent laid open publication No. 2,163,619A. in which an error concealment is primarily executed at an error correcting section 2 with regard to a signal output from a digital video tape recorder( DVTR ) 1, and the error is amended during passing through an error amending section 3 consisting of a temporal error amending section 3a and a spatial error amending section 3b for the pixel where the correction of error is impossible.
Operation of the error amending section 3 will be described with reference to the FIG. 2 which shows a sample arrangement at positions corresponding to two consecutive frames in a television video signal. Here, S1 to S8 and EO represent nine pixels of a present frame, and S1D to S8D and EOD represent pixels of past frame respectively.
The pixels S1 to S8 of the present frame are subtracted from the corresponding pixels of the past frame to judge whether or not the temporal amendment is suitable, by the subtracters 27 to 34 of FIG. 3 which is the detailed block diagram of the system shown in FIG. 1. Then the differential signals of the subtracters 27 to 34 are input to Programmable Read Only Memories ( PROMs ) 35 to 42, and the signals input from these subtracters 27 to 34 are scaled at the PROMs 35 to 42.
At this moment, the PROMs 35 to 42 refer an error flag coming out of OR gate 44, and this error flag becomes "1" when any error is present in any pixel of the pixel S1 or the pixel S1D and when the error flag is "1", the PROM 35 outputs "0". Remaining PROMs 36 to 42 also operate by same method.
The outputs of the PROMs 35 to 42 are added by an adder 46 and input to a PROM 47 and thereby scaled.
The output of the prom 47 is input to a PROM 48 and at this moment, it is judged whether or not it is substituted by a pixel EO delayed by one frame. Here,three conditions are involved for executing the temporal error amendment. First, the error flag should inform that present pixel EO is an error, second a pixel EO delayed by one frame has to have no error, and third the output of the PROM 47 should not exceed a predetermined limitation.
Since the conventional error concealment system conceals an error on the pixel by pixel, it has a disadvantage that it can be applied only to the video compressing systems such as pulse code modulation( PCM ) or differential pulse code modulation( DPCM ). Because the video compressing system for digital storage media( DSM ) presently offered by MPEG encodes by using a block-based coding system such as motion compensation( MC ) and discrete cosine transform( DCT ), the error is produced in a block units. Also since the motion compensation is executed, the error in the motion vector should be considered as well as in the picture data. Therefore the error concealment system in pixel units as described above becomes impossible to use.
And, as described above, the error concealment system in pixel units can not share the circuits used in the conventional coding system, and since a complicated circuit is involved, there is a problem of high cost.
The video signal compression method of MPEG will be described as follows. One frame is divided into units called macro blocks and transmitted in macro block units. Also there is a picture mode which discriminates whether the transmitted frame is either an intra-mode or a non-infra mode (e.g., a predictive mode and bidirectional mode). The picture mode tells whether the pixels of other frame needs to be considered or not. In macro block units, other data, i.e., data such as the motion vector or picture information, are encoded all together and transmitted.
In detail, the frame is classified into an intra frame, a predictive frame and a bidirectional frame.
The intra frame means a frame was DCT coded by considering only a redundancy within its frame without considering the differences from other frames;
the predictive frame means a frame which has only forward motion vectors, and DCT coding is applied to the difference from the corresponding prediction block in the prior intra frame or in the predictive frame referred by the motion vectors: and PA1 the bidirectional frame means a frame for coding the block data which is either interpolated from both the past and future frame by utilizing both of the forward and reverse motion vectors from the past frame and the future frame or substituted from or either of the two frames by utilizing the motion vectors of one direction. PA1 Si is a pixel value of the block to be presently interpolated, PA1 S1 is a corresponding pixel value of the past frame, x,y represent respectively the position of the pixel. PA1 a step for judging whether or not an error is present in a mode data among an information signal of a macro block unit, PA1 a step for judging whether or not the mode is an intra mode in case when an error is not present in the mode data, PA1 a step for checking the number of motion vectors having no error in case of non-intra mode, PA1 a step for substituting the macro block designated by the motion vector when the number of error-free motion vector is 1, and for interpolating the macro blocks designated by the motion vectors when the number is 2, and PA1 a step for interpolating the past and future intra frame or predictive frame being present at a same position corresponding to the macro block with an error in case where an error is present in mode data or the macro block is an intra mode or in case where an error is present in all motion vectors. PA1 digital storage media, PA1 an error correcting section correcting the error in the digital video signal from said DSM and outputting the error signal to other output terminal in case when the error correction is impossible, PA1 a decoding section for decoding the digital video signal which is connected to an input terminal of said error correcting section and error-corrected, and PA1 an error concealment control section which receives said error signal of said error correcting section and outputs the error concealment control signal to said decoding section so as to conceal a bidirectional frame corresponding to said error signal in accordance with a mode information generated at said decoding section.
Since the bidirectional frame and the predictive frame successively following the intra frame are encoded based on the intra frame, in case any error occur in the intra frame, the bidirectional frame and the predictive frame are reconstructed on the basis of the erroneous intra frame during decoding. Therefore the error in the intra frame is propagated through the future frames. And, also in case when the error is occurred in the predictive frame, as described above, the past and future bidirectional frame and the predictive frame can not be reconstructed perfectly.
On the other hand, in case when an error is generated in the bidirectional frame it has a tendency to be limited to its own frame without being propagated to other frames.
When the error is generated in this bidirectional frame, particularly when the error cannot be corrected by the error correction decoder because its error correcting capability is exceeded, it should be compensated in the macro block unit the erroneous macro block can be concealed by interpolation, utilizing the macro block of past or future intra frame or predictive frame which is present at same position as a block in which an error is generated.
The Following can be used for such interpolation. ##EQU1##
This represents a case that the predictive frame or the intra frame is present at 1 and (m1) on a time axis and the macro block of the bidirectional frame to be interpolated is present at i position.
When the macr blocks in the frames 1,i, m+1 are in the same spatial position it becomes as x1=x2=x3, y1=y2=y3